Skip to content

Add ZeroGPU connector - #9

Merged
aezell merged 3 commits into
superfly:mainfrom
zerogpu:add-zerogpu-connector
Oct 30, 2026
Merged

aezell merged 3 commits into
superfly:mainfrom
zerogpu:add-zerogpu-connector

Conversation

@nishithatanukunuri

@nishithatanukunuri nishithatanukunuri commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

owner: vendor

Adds connectors/zerogpu, which lets a Sprite run OpenAI-compatible inference on ZeroGPU's open-weight models and specialized small language models (ZLMs) for generation, classification and extraction, without holding the ZeroGPU API key. Submitted by ZeroGPU.

Routes

Route Docs
POST /v1/chat/completions https://docs.zerogpu.ai/api-reference/chat-completions
POST /v1/responses https://docs.zerogpu.ai/api-reference/responses
GET /v1/me (probe) https://docs.zerogpu.ai/integrations/sprites#how-key-validation-works (our Sprites guide, published when this connector ships; until then the Probe section below and connectors/zerogpu/README.md document the endpoint)

Model IDs used in the examples: https://docs.zerogpu.ai/docs/model-catalog

Both routes support SSE ("stream": true). Left out: model listing, embeddings, moderations, audio, batches, files, API-key management, billing and account routes.

Credentials

  • One key, sent as x-api-key.
  • Each key belongs to exactly one project, and the API derives the project from the key. The API ignores the x-project-id header, so it is not forwarded.

Probe

GET /v1/me returns the key, project and organization the key belongs to ({"object": "api_key", "id", "project_id", "organization_id"}). We added this authenticated read-only endpoint for the connector, because GET /v1/models is public and can't tell a good key from a bad one.

  • It is free: no inference, no balance check, no free-trial call, and no usage is recorded. It's sent with Cache-Control: no-store.
  • It reads the key's row on every call, so a revoked key fails immediately.
  • A wrong or revoked key gets 403 with the plain-text body Invalid credentials, hence the errors rule. A missing key gets 401.
  • The probe asserts object = api_key, non-empty id, project_id and organization_id, and an absent error.

forward_response_headers: x-request-id, which is on every response and which support uses to trace a request, and retry-after, which comes with 429s.

Icons

Both icons come from ZeroGPU's own logo, the dark favicon our dashboard serves at https://platform.zerogpu.ai/assets/favicon_dark.svg. ZeroGPU owns the mark and submits it for this listing.

  • icon.svg is the logo (black disc, green ring and halves) without the square favicon background, cropped to the circle.
  • mark.svg is the ring and halves alone, with every fill set to currentColor.

Validation

  • node scripts/check-connectors.mjs: 14 connector(s) ok (Node 22)
  • npx --yes ajv-cli@5 validate --spec=draft2020 --strict=false -s v1/schema.json -d connectors/zerogpu/connector.json: valid
  • node scripts/probe.mjs zerogpu --key-env ZEROGPU_API_KEY with a valid key:
    GET https://api.zerogpu.ai/v1/me (timeout 10000 ms)
    HTTP 200
      ok   ["object","eq","api_key"]
      ok   ["id","text"]
      ok   ["project_id","text"]
      ok   ["organization_id","text"]
      ok   ["error","absent"]
    probe ok
    
  • With a wrong key: HTTP 403 → FAIL invalid_api_key. A revoked key also gets 403 (checked on our staging and dev environments).
  • Recording: fixtures/zerogpu/probe_200.json. Credential headers are removed, and the three IDs in the body are replaced with placeholder UUIDs.
  • Our Cloudflare WAF accepts the gateway's Sprites-API-Gateway user agent.
  • Each example in the manifest returned 200 with that user agent, and SSE streaming worked on both routes. An unknown model returns 404 model_not_found.

OpenAI-compatible inference on ZeroGPU's open-weight models and
specialized small language models (ZLMs), forwarding only
POST /v1/chat/completions and POST /v1/responses.

The API has no authenticated read-only endpoint (GET /v1/models is
public), so the probe is a minimal chat completion on our cheapest ZLM;
the Billing notice discloses the charge.
@aezell

aezell commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

@nishithatanukunuri This is currently marked as draft but so far it looks good. I can take another look if you end up making changes before marking it ready.

ZeroGPU now has an authenticated, read-only key identity endpoint, so
the probe no longer runs a paid chat completion. GET /v1/me is free,
does not touch the balance or free trial calls, and rejects revoked keys
immediately. It is added to the allowed routes, and the Billing and
Scope notices are updated to match.
@nishithatanukunuri
nishithatanukunuri marked this pull request as ready for review October 30, 2026 16:38
@nishithatanukunuri

Copy link
Copy Markdown
Contributor Author

Thanks @aezell! I made one change since your look, and it's now ready for review.

The probe now uses a new GET /v1/me endpoint on our API instead of a chat completion. It returns the key, project and organization the key belongs to. It's free and read-only: it runs no inference, doesn't check the balance and records no usage. It also reads the key's row on every call, so a revoked key fails right away. GET /v1/me is added to the allowed routes, and the Billing and Scope notices are updated to say the key check is free.

The probe passes against production with a valid key, and a wrong key is classified as invalid_api_key. The fixture is re-recorded with placeholder IDs. GET /v1/me is documented in connectors/zerogpu/README.md. The docs link in the description goes live on docs.zerogpu.ai when the connector ships.

Happy to change anything else.

@aezell

aezell commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

@nishithatanukunuri The CI is misbehaving. I'm going to try closing and reopening the PR. Wanted to warn you if you saw notifications.

@aezell aezell closed this Oct 30, 2026
@aezell aezell reopened this Oct 30, 2026
@aezell
aezell merged commit 38e3d88 into superfly:main Oct 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants